IEEE TNN A172Rev K Nearest Neighbours Directed Noise Injection in Multilayer Perceptron Training

IEEE TNN A172Rev K Nearest Neighbours Directed Noise Injection in Multilayer Perceptron Training M. Skurichina1, .Raudys2 and R.P.W. Duin1 1Pattern Recognition Group, Department of Applied Physics, Delft University of Technology, P.O. Box 5046, 2600GA Delft, The Netherlands. E-mail: [email protected], [email protected] 2Department of Data Analysis, Institute of Mathematics and Informatics, Akademijos 4, Vilnius 2600, Lithuania. Email: [email protected] Abstract The relation between classifier complexity and learning set size is very important in discriminant analysis. One of the ways to overcome the complexity control problem is to add noise to the training objects, increasing in this way the size of the training set. Both, the amount and the directions of noise injection are important factors which determine the effectiveness for classifier training. In this paper the effect is studied of the injection of Gaussian spherical noise and k nearest neighbours directed noise on the performance of multilayer perceptrons. As it is impossible to provide an analytical investigation for multilayer perceptrons, a theoretical analysis is made for statistical classifiers. The goal is to get a better understanding of the effect of noise injection on the accuracy of sample based classifiers. By both, empirical as well as theoretical studies, it is shown that the k nearest neighbours directed noise injection is preferable over the Gaussian spherical noise injection for data with low intrinsic dimensionality.